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ABSTRACT 


The  purpose  of  this  report  is  to  document  the  transition  of  the  Complex  Visual  Task  (CVT) 
from  its  original  format  to  a  computer-based  (Windows  environment)  performance  test.  We 
provide  initial  background  information  on  the  test  as  well  as  a  detailed  description  of  the 
computerized  version.  Issues  such  as  navigation  through  the  program  and  scoring  the  test  are 
also  addressed.  Sample  input  and  output  fDles  necessary  for  test  administration  are  included.  The 
report  concludes  with  a  brief  description  of  an  application  of  the  CVT  in  an  experimental  selection 
test  battery  for  Landing  Craft  Air  Cushion  (LCAC)  vehicle  navigators. 


1 


Acknowledgments 


We  relied  on  many  portions  of  the  original  report  by  LCDR  T.  Morrison  entitled  Complex 
Vi^al  Information  Processing^  Test  for  Predicting  Navy  Primary  Flight  Training  Success 
(Morrison,  1988).  We  wish  to  acknowledge  the  efforts  of  Dr.  R.J.  Wherry  in  the  original 
development  of  the  Complex  Visual  Display  Task.  We  gratefully  acknowledge  the  guidance  and 
organization  of  LCDRP.M.  Holmes,  Dr.  T.  Nontasak,  and  Mr.  A.  Chapman.  We  also  appreciate 
the  efforts  of  Dr.  D.  Blower,  Mr.  A.  Chapman,  and  LT  S.  Biggerstaff  for  their  critical  review. 
Special  acknowledgment  is  given  to  Ms.  M.  Thomas  and  ENS  B.  Williamson  for  subject  testing 
and  database  management.  We  also  thank  Ms.  K.  Mayer  for  her  editorial  contributions. 


u 


INTRODUCTION 


The  original  form  of  the  Complex  Visual  Task  (CVT)  was  based  on  a  comprehensive  task 
analysis  of  the  component  skills  needed  to  extract  information  from  visual  displays,  tasks 
underlying  the  jobs  performed  by  both  naval  aviators  and  naval  flight  officers  (NFOs).  The  CVT 
was  originally  designed  to  accurately  reflect  the  cockpit  environment  and  to  measure  individual 
visual  information  processing  capabilities  (Morrison,  1988).  Specifically,  the  test  assesses  an 
individual's  ability  to  encode  verbal  information  and  manipulate  pictorial  displays  in  short-term  or 
working  memory.  These  skills  are  of  increasing  importance  as  the  array  of  instrumentation  within 
an  aircraft  has  escalated  and  become  more  complex  over  the  years.  In  fact,  Morrison  (1988) 
found  that  the  CVT  was  predictive  of  success  in  Navy  primary  flight  training.  Subsequent  to  the 
work  with  aviation  personnel,  the  CVT  was  adapted  by  the  Naval  Aerospace  Medical  Research 
Laboratory  (NAMRL)  for  use  in  an  experimental  selection  test  batteiy  for  the  Landing  Craft  Air 
Cushion  (LCAC)  vehicle  navigator  crew  position.  The  LCAC  training  command  experienced 
high  attrition  rates  in  its  navigator  program  in  the  last  few  years.  Task  analysis  of  the  LCAC 
operational  environment  showed  that  the  navigator  position  encompasses  many  of  the  same  skill 
requirements  of  the  NFO  (Hunt,  Linville,  Stuster,  Schneider  &  Braun,  1993). 

Originally,  the  CVT  required  an  Apple  II  computer  and  a  35-mm  slide  projector.  The 
questions  and  answers  were  displayed  on  an  Apple  II  monitor,  with  the  slides  displayed  on  the 
adjacent  projector.  Subjects  were  instructed  to  read  questions  on  the  monitor,  press  the  mter 
key,  and  the  tactical  display  would  be  shown  on  the  projector  located  to  the  right  of  the 
computer.  The  newer  version  of  the  CVT  was  developed  by  NAMRL  for  any  IBM-PC 
compatible  computer  running  Microsoft®  Windows  3.1™  or  higher.  This  Window’s  version  of 
the  CVT  eliminates  the  need  for  a  projector  by  centralizing  the  questions,  displays,  and  response 
feedback  on  the  computer  monitor.  The  upgrade  of  the  CVT  has  increased  both  the  accessibility 
as  well  as  the  cost-effectiveness  of  the  test.  The  computerized  CVT  can  be  easily  distributed  on 
floppy  disk  and  run  on  a  compatible  computer. 

To  port  the  CVT  to  an  IBM-compatible  PC,  attributes  from  each  object  on  a  slide  were 
collected  to  retain  the  integrity  of  the  original  test  questions.  The  attributes  recorded  included  the 
object’s  location  (i.e.,  x  and  y  coordinates),  angle  (45,  90, 180  deg,  etc.),  shape  (triangle, 
rectangle,  pentagon),  size  (small,  medium,  large),  and  color  (red,  green,  white).  To  collect  the  x 
and  y  coordinates,  a  transparent  graph  paper  overlay  was  positioned  over  the  original  slide 
projection  to  correctly  map  the  inner  triangle’s  point  of  direction  for  each  object.  The  additional 
attributes  were  recorded  for  each  object  from  the  original  CVT  slides.  As  the  slide  information 
was  coded  to  a  Windows  environment,  each  slide’s  accuracy  was  verified  by  measuring  the  output 
of  the  VGA  monitor  against  the  original  slide  specifications.  The  objects  on  the  monitor  output 
were  continuously  readjusted  to  match  their  original  placement  and  direction. 

CVT  TEST  DESCRIPTION 

The  original  test  is  comprised  of  120  questions  and  typically  requires  60-90  min  to  complete. 
The  computerized  version  of  the  CVT  provides  a  comprehensive  set  of  instructions  incorporating 
color  graphics  to  provide  actual  examples  and  additional  clarification.  In  the  instruction  section, 
the  subject  is  informed  that  triangles  represent  airplanes,  rectangles  represent  aircraft  carriers,  and 
pentagons  represent  destroyers.  The  subject  must  memorize  the  representations  to  respond 
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accurately  to  several  test  questions.  In  terms  of  test-taking  strategies,  the  subject  is  instructed  to 
perform  the  speed  and  accuracy  trade-off  by  completing  the  questions  “as  quickly  and  as 
accurately  as  possible.” 

Navigating  Through  the  Test 

At  the  onset  of  the  test,  the  subject  is  presented  with  a  test  question.  After  reading  the 
question,  the  subject  prompts  the  tactical  display  by  pressing  the  enter  key.  The  display  contains 
a  circle  with  crosshairs  and  a  number  of  objects  of  various  shapes,  sizes,  colors,  quadrant  location, 
and  direction.  The  subject  is  instructed  to  pay  close  attention  to  the  objects  and  their  various 
attributes.  The  bottom  of  the  display  contains  instructions  and  a  a  field  for  entering  the  answer. 
For  example,  the  subject  is  reminded  to  enter  ‘1’  for  ‘true’  responses,  2  for  false  responses,  or 
a  number  for  the  quantitative  questions.  For  example:  More  than  7  objects  are  in  the  right 
portion  of  the  screen  (True  or  False)?  (Fig.  1).  The  correct  answer  is  false. 


Question  Number  1 

More  than  7  objects  arc  in  the  right  portion  of  the 
screen  (True  or  False)? 


Figure  1.  CVT  Question  Display 

The  subject  is  required  to  encode  the  quantity  and  location  of  the  objects  on  the  display  in  order 
to  respond  to  the  question. 
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Test  questions  are  arranged  in  increasing  difficulty.  The  determination  of  the  difficulty  is 
presented  in  the  section  detailing  the  CVT’s  use  in  an  experimental  test  battery.  An  example  of  a 
“hard”  question  is:  No  more  than  3  medium-sized  objects  are  in  the  upper  portion  of  the  screen 
outside  the  circle?  This  question  requires  the  subject  to  recall  the  quantity  of  three,  the  shape, 
and  a  particular  area  of  the  screen  outside  the  circle.  The  examinee  will  press  the  mter  key  to 
prompt  the  tactical  display  (Fig.  2) . 


Figure  2.  CVT  Tactical  Display. 

When  the  subject  responds,  feedback  is  provided  at  the  bottom  of  the  display.  By  pressing  the 
enter  key,  the  subject  prompts  the  next  question. 

Scoring  the  Test 

The  first  test  performance  measure  is  question  accuracy.  We  also  record  two  reaction  times 
for  each  question.  The  first  reaction  time,  referred  to  as  question  comprehension  time,  is  the 
duration  between  question  presentation  and  the  subject’s  response  (i.e.,  prompting  the  tactical 
display).  The  second  reaction  time,  answer  reaction  time,  is  the  duration  between  the 
presentation  of  the  tactical  display  and  the  subject’s  response. 

THE  OUTPUT  FDLE 

In  the  first  column  of  the  sample  output  file,  Q#,  identifies  the  question  number  (Table  1). 
The  second  column,  question  time,  indicates  the  question  comprehension  time  in  seconds.  The 
third  column,  answer  time,  indicates  the  answer  reaction  time  in  seconds.  The  fourth  column 
contains  the  subject’s  actual  input.  The  fifth  column  contains  the  correct  answer  to  the  test 
question.  The  sixth  column  indicates  whether  the  subject  answered  the  question  correctly  or 
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incorrectly.  A  correct  response  is  coded  as  a  1,  an  incorrect  response  as  a  zero.  The  seventh 
column  tallies  the  number  of  correct  responses  and  indicates  the  cumulative  score  for  the  entire 
test.  The  output  files  are  labeled  as  an  *.out  extension. 

At  the  bottom  of  each  test  file  is  a  section  which  summarizes  the  subject’s  overall  test  results. 
Mean  reaction  times  are  individually  computed  for  responses  answered  correctly  and  incorrectly, 
as  well  as  an  overall  average  (Table  2).  The  total  column  displays  the  number  of  correct, 
incorrect,  and  total  responses. 


Table  1.  CVT  Sample  Output  FUe 


0# 

Question 

Time 

Answer 

Time 

Subjet^^s 

Afiswef 

Coftec^ 

Answer 

Correct 
Score . . 

-  Cam, 

.  Score  • 

1 

20.60 

6.49 

T 

T 

1 

1 

2 

9.00 

5.79 

T 

F 

0 

1 

3 

5.05 

9.56 

6 

7 

0 

1 

4 

10.38 

7.67 

4 

4 

1 

2 

5 

23.35 

11.43 

F 

T 

0 

2 

Av€^nge  Question  Time 

Avemge  AnswerTime 

Total 

Correct 

15.49 

7.08 

2 

Incorrect 

12.46 

8.93 

3 

Overall 

13.68 

8.19 

5 

THE  INPUT  FILE 

To  reiterate,  the  CVT  is  composed  of  tactical  displays  that  depict  objects  of  various  attributes. 
Each  slide  contains  between  12-20  objects.  To  display  each  object  on  the  monitor,  the  data 
collected  on  its  attributes  (shape,  size,  color,  angle,  and  location)  were  coded  for  a  Windows 
environment.  The  description  of  the  sample  input  file  is  described  in  Fig.  3. 


CVTT 

<QT=F> 

<N=1> 

<T> 

Exactly  5  objects  are  in  the  right  portion  of  the  screen  (True  or  False)? 
</T> 

<J  2  2  3  45  -25  97> 

<J  1  2  1  45  78  94>  _ _ _ 
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<J  1  2  2  45  -102  50> 

<J  1  2  1  45  5  27> 

<J  3  2  1  45  42  -7> 

<J  1  2  2  180  59  -42> 

<J  1  2  2  45  -34  -77> 

<J3  2  1  180  111  -55> 

<J3  22  180  21  -71> 

<J  1  22  180-123  -93> 

<R> 

Exactly  6  fit  the  description  given. 
</R> 

<A=6> 

</Q> 

</.> 


Figure  3.  Sample  CVT  Input  File 

The  first  line  contains  CVTT,  which  is  a  tag  that  identifies  the  file  as  an  input  test  file.  The 
next  line  contains  the  question  type  tag  <QT=F>  followed  by  the  equal  sign  indicating  the  correct 
response.  Other  question  types  include  <QT=T>  used  to  identify  the  question  as  a  true/false  t5q)e 
question  with  a  correct  answer  of  true,  and  <QT=N>,  to  identify  the  question  as  a  quantitative 
question  with  a  correct  numerical  answer.  The  following  line  <N-I>  identifies  this  question  as 
question  number  1  in  the  CVT  test.  The  next  three  lines  contain  the  question  text.  The  tag, 

<T>,  indicates  that  the  beginning  of  the  question  text  will  appear  on  the  subsequent  line. 
Following  the  question  text  is  the  tag  </T>,  which  denotes  the  end  of  the  question  text. 

The  next  10  lines  describe  the  various  object  attributes,  with  each  line  representing  one 
object.  Each  line  begins  with  <J,  followed  by  the  object  data,  followed  by  >  that  marks  the  end 
of  that  object’s  description.  On  the  first  line,  the  object  data  are  shown  as  <J  2  2  3  45  -25  97>. 
The  object  data  are  arranged  in  the  following  order;  shape,  size,  color,  angle,  and  x,  y 
coordinates.  Each  object  is  one  of  three  possible  shapes:  triangles,  rectangles  or  pentagons  and  is 
represented  as  the  numbers  1,  2,  and  3,  respectively.  Similarly,  each  object  is  one  of  three 
possible  sizes:  small,  medium,  or  large,  and  is  indicated  as  1, 2,  and  3,  respectively.  Each  object 
is  one  of  three  possible  colors,  red,  green,  or  white,  and  are  indicated  as  1,  2,  and  3,  respectively. 
The  next  data  point  indicates  the  angle  of  the  object.  The  most  common  angles  included  0,  45, 

90,  180,  and  270  deg  although  adjustments  were  made  to  ensure  the  direction  of  the  object  on  the 
monitor  was  similar  to  the  object  on  the  slide.  The  angles  increase  in  magnitude  in  a  clockwise 
manner.  The  final  two  items  of  data  indicate  the  location,  or  x  and  y  coordinates,  of  each  object 
based  on  the  location  of  the  point  of  the  inner  triangle.  For  example,  <J  2  2  3  45  -25  97> 
indicates  that  this  object  is  a  medium  white  rectangle  with  a  45®  angle  at  -25,  97  on  the  tactical 
display. 

The  next  three  lines  in  the  original  sample  input  file  describe  the  feedback  text  displayed  after 
the  subject’s  response.  The  tag  <R>  informs  the  CVT  software  that  the  response  text  will  be  on 
the  following  line,  and  the  tag  <\R>  indicates  the  end  of  the  response.  In  the  example,  after  the 
subject  responds,  the  text  “Exactly  6 fit  the  description  given”  is  displayed.  After  this  line,  the 
tag  <A=6>  indicates  that  the  numerical  answer  to  the  question  is  six.  The  last  line  of  the  example 
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shows  a  tag  </Q>,  indicating  the  end  of  question  number  one.  The  input  file  format  described  in 
this  section  continues  for  each  question  until  an  end  of  test  tag  is  located.  The  end  of  test  tag  is 
indicated  by  <./>. 


TEST  OPERATION  FILES 


Administration  Section 

The  following  list  indicates  the  files  necessary  to  administer  the  CVT.  Researchers  at  the 
NAMRL  attempted  to  adjust  the  length  of  the  CVT  because  it  will  be  one  of  several  tests  in  an 
experimental  test  battery.  The  initial  test  data  collected  was  based  on  the  first  60  questions  and 
the  second  60  questions,  which  explains  the  various  test  files.  The  process  used  to  select  60 
appropriate  questions  is  discussed  in  a  later  section.  The  following  description  provides  an 
outline  of  the  directory  and  file  setup  used  to  administer  the  pilot  version  of  the  CVT  to  test 
subjects. 


Contents  of  \cvt 

-demoMnput 

cvtl-60.tml 

Input  test  file  that  contains  questions,  answers,  and 
object  data  for  slides  1  -  60. 

cvt61-.tml 

Input  test  file  that  contmns  questions,  answers,  and 
object  data  for  slides  61  - 120. 

navig.tml 

Input  test  file  that  contmns  questions,  answers,  and 
object  data  for  slides  selected  for  experimental  test  battery. 

demo.tml 

Input  test  file  that  contains  questions,  answers,  and 
object  data  for  10  slides  used  for  demonstration  purposes. 

Contents  of  \cvt-demo\output 

*  out  Any  file  with  an  .out  extension  is  a  subject’s  test  file. 

Software  Engineer  Section 

To  ensure  portability  with  future  versions  of  the  CVT,  we  recommend  using  the 
Borland  C++  compiler  (version  4.0  or  higher). 

Contents  of  \cvt-demo\source  C++  language  source  files 

cvt.cpp  Source  code  for  CVT  test, 

cvt.h  Header  file  for  cvt.cpp. 

instruct.cpp  Source  code  for  Instructions  section. 
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instruct.h 

waiter.cpp 

waiter.h 

adminxpp 

admin.h 

info.cpp 

windows.h 

cvt.ide 

cvt.rc 

cvt-lbl.bmp 

pgdown.bmp 

center.bmp 

cvtcl.bmp 

cvt-lbl2.bmp 

nsew.bmp 

ultest.bmp 

urtest.bmp 

lrtest.bmp 

lltest.bmp 

shapes.bmp 

triangle.bmp 

pentagon.bmp 

rctangle.bmp 

cvt.ico 


Header  file  for  instruct.cpp. 

Source  code  for  waiting  between  the 
Administration  and  Instruction  sections. 

Header  file  for  waiter.cpp. 

Source  code  for  test  file  and  directory  setup. 

Header  file  for  admin,  cpp. 

Source  code  file  for  Subject  Number  section. 

Header  file  for  Windows  API  calls. 

Project  file  for  CVT  (Borland  €++  4.0  compiler). 

Resource  file  for  CVT  questions. 

CVT  introduction  label. 

Page  down  label  for  the  CVT  instructions. 

Instruction  example  showing  the  center  of  the  circle. 

Tactical  display  example  slide. 

Heading  for  CVT  instructions. 

Instruction  example  showing  north,  south, 
east,  and  west  on  the  tactical  display. 

Instruction  example  showing  the  upper  left 
section  of  the  tactical  display. 

Instruction  example  showing  the  upper  right 
section  of  the  tactical  display. 

Instruction  example  showing  the  lower  right 
section  of  the  tactical  display. 

Instruction  example  showing  the  lower  left 
section  of  the  tactical  display. 

Instruction  example  showing  each  shape’s 
size  and  color. 

Instruction  example  slide  of  triangles. 

Instruction  example  slide  of  pentagons. 

Instruction  example  slide  of  rectangles. 

Icon  for  the  CVT. 


Contents  of  \cvt-denio\instr 

cvtinst.txt  Instruction  file  for  the  CVT. 


Contents  of  \cvt-denio\bin 

cvt.exe  Executable  program  for  the  CVT. 

Contents  of  Windows 

cvt.ini  Initialization  file  that  contains  the  settings  for 

the  directories. 
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MINBVIUM  HARDWARE  CONFIGURATION 

386-25  MHZ  IBM-conipatible  PC  with  a  minimum  of: 

4  MB  RAM  memory 
1.44  MB  3.5”  floppy  disk  drive 
20-MB  hard  disk  drive 
VGA  video  card 

15”  VGA  compatible  color  monitor 

Uninterruptable  Power  Supply 

Numerikeys  Ke)rpad  System 

APPLICATION  OF  THE  COMPUTERIZED  CVT 

Morrison  (1988)  demonstrated  the  predictive  validity  of  the  CVT  in  the  selection  of  naval 
aviators  and  NFOs.  Due  to  the  similar  nature  of  the  task  demands,  the  researchers  at  the 
NAMRL  incorporated  the  computerized  version  of  the  CVT  into  an  experimental  test  battery  to 
be  used  in  the  selection  of  Landing  Craft  Air  Cushion  (LCAC)  vehicle  navigators.  For  purposes 
of  a  subtest  in  the  experimental  test  battery,  the  length  of  the  CVT  was  decreased  from  the 
original  120  to  60  questions. 

To  select  the  appropriate  test  question  bank,  performance  data  were  collected  from  67 
aviators  and  flight  officers.  The  data  were  analyzed  by  determining  the  level  of  difficulty  of  each 
question  independently.  The  criteria  for  the  level  of  difficulty  were  based  on  the  percentage  of  the 
sample  with  the  correct  answer  in  relation  to  both  of  the  reaction  times.  The  test  items  were 
classified  as  easy,  medium,  or  difficult.  Fifteen  ‘easy’  questions,  30  ‘medium  questions,  and  15 
‘difficult’  questions  were  chosen  for  the  shorter  version  of  the  test.  The  revised  test  requires 
approximately  30-45  min  to  complete  and  was  incorporated  into  the  LCAC  Navigator  Selection 
System.  The  system,  which  includes  the  CVT  and  other  computer-based  tests  developed  at 
NAMRL  (Chapman  &  Nontasak,  1996,  in  press;  Helton,  Nontasak,  &  Dolgin,  1992;  Dolgin, 
Street,  Nontasak,  Blower,  &  Travis,  1992)  is  currently  being  used  operationally  to  test  LCAC 
Navigator  candidates  at  NAMRL  and  provide  assignment  recommendations  to  the  Bureau  of 
Nav^  Personnel. 
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